14 research outputs found
Transparency in Complex Computational Systems
Scientists depend on complex computational systems that are often ineliminably opaque, to the detriment of our ability to give scientific explanations and detect artifacts. Some philosophers have s..
Transparency in Complex Computational Systems
Scientists depend on complex computational systems that are often ineliminably opaque, to the detriment of our ability to give scientific explanations and detect artifacts. Some philosophers have suggested treating opaque systems instrumentally, but computer scientists developing strategies for increasing transparency are correct in finding this unsatisfying. Instead, I propose an analysis of transparency as having three forms: transparency of the algorithm, the realization of the algorithm in code, and the way that code is run on particular hardware and data. This targets the transparency most useful for a task, avoiding instrumentalism by providing partial transparency when full transparency is impossible
Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?
As the scope of machine learning broadens, we observe a recurring theme of
algorithmic monoculture: the same systems, or systems that share components
(e.g. training data), are deployed by multiple decision-makers. While sharing
offers clear advantages (e.g. amortizing costs), does it bear risks? We
introduce and formalize one such risk, outcome homogenization: the extent to
which particular individuals or groups experience negative outcomes from all
decision-makers. If the same individuals or groups exclusively experience
undesirable outcomes, this may institutionalize systemic exclusion and
reinscribe social hierarchy. To relate algorithmic monoculture and outcome
homogenization, we propose the component-sharing hypothesis: if decision-makers
share components like training data or specific models, then they will produce
more homogeneous outcomes. We test this hypothesis on algorithmic fairness
benchmarks, demonstrating that sharing training data reliably exacerbates
homogenization, with individual-level effects generally exceeding group-level
effects. Further, given the dominant paradigm in AI of foundation models, i.e.
models that can be adapted for myriad downstream tasks, we test whether model
sharing homogenizes outcomes across tasks. We observe mixed results: we find
that for both vision and language settings, the specific methods for adapting a
foundation model significantly influence the degree of outcome homogenization.
We conclude with philosophical analyses of and societal challenges for outcome
homogenization, with an eye towards implications for deployed machine learning
systems.Comment: Published at NeurIPS 2022, presented at EAAMO 202
Ecosystem Graphs: The Social Footprint of Foundation Models
Foundation models (e.g. ChatGPT, StableDiffusion) pervasively influence
society, warranting immediate social attention. While the models themselves
garner much attention, to accurately characterize their impact, we must
consider the broader sociotechnical ecosystem. We propose Ecosystem Graphs as a
documentation framework to transparently centralize knowledge of this
ecosystem. Ecosystem Graphs is composed of assets (datasets, models,
applications) linked together by dependencies that indicate technical (e.g. how
Bing relies on GPT-4) and social (e.g. how Microsoft relies on OpenAI)
relationships. To supplement the graph structure, each asset is further
enriched with fine-grained metadata (e.g. the license or training emissions).
We document the ecosystem extensively at
https://crfm.stanford.edu/ecosystem-graphs/. As of March 16, 2023, we annotate
262 assets (64 datasets, 128 models, 70 applications) from 63 organizations
linked by 356 dependencies. We show Ecosystem Graphs functions as a powerful
abstraction and interface for achieving the minimum transparency required to
address myriad use cases. Therefore, we envision Ecosystem Graphs will be a
community-maintained resource that provides value to stakeholders spanning AI
researchers, industry professionals, social scientists, auditors and
policymakers.Comment: Authored by the Center for Research on Foundation Models (CRFM) at
the Stanford Institute for Human-Centered Artificial Intelligence (HAI).
Ecosystem Graphs available at https://crfm.stanford.edu/ecosystem-graphs